Belief Propagation Algorithms for Mean-Field Combinatorial Optimisations
نویسنده
چکیده
We study combinatorial optimisation problems on graphs in the mean-field model, which assigns independent and identically distributed random weights to the edges of the graph. Specifically, we focus on two generalisations of minimum weight matching on graphs. The first problem of minimum cost edge cover finds application in a computational linguistics problem of semantic projection. The second problem of minimum cost many-to-one matching appears as an intermediate optimisation step in the restriction scaffold problem applied to shotgun sequencing of DNA. For the minimum cost edge cover on a complete graph on n vertices, where the edge weights are independent exponentially distributed random variables, we show that the expectation of the minimum cost converges to a constant as n→∞. For the minimum cost many-to-one matching on an n × m complete bipartite graph, scaling m as dn/αe for some fixed α > 1, we find the limit of the expected minimum cost as a function of α. For both problems, we show that a belief propagation algorithm converges asymptotically to the optimal solution. The belief propagation algorithm yields a near optimal solution with lesser complexity than the known best algorithms designed for optimality in worst-case settings. Our proofs use the machinery of the objective method and local weak convergence, which are ideas developed by Aldous for proving the ζ(2) limit for the minimum cost bipartite matching. We use belief propagation as a constructive proof technique to supplement the objective method. Recursive distributional equations (RDEs) arise naturally in the objective method approach. In a class of RDEs that arise as extensions of the minimum weight matching and travelling salesman problems, we prove existence and uniqueness of a fixed point distribution, and characterise its domain of attraction.
منابع مشابه
Belief Propagation, Mean-field, and Bethe approximations
This chapter describes methods for estimating the marginals and maximum a posteriori (MAP) estimates of probability distributions defined over graphs by approximate methods including Mean Field Theory (MFT), variational methods, and belief propagation. These methods typically formulate this problem in terms of minimizing a free energy function of pseudomarginals. They differ by the design of th...
متن کاملUsing Combinatorial Optimization within Max-Product Belief Propagation
In general, the problem of computing a maximum a posteriori (MAP) assignment in a Markov random field (MRF) is computationally intractable. However, in certain subclasses of MRF, an optimal or close-to-optimal assignment can be found very efficiently using combinatorial optimization algorithms: certain MRFs with mutual exclusion constraints can be solved using bipartite matching, and MRFs with ...
متن کاملSurvey Propagation: Iterative Solutions to Constraint Satisfaction Problems
Iterative algorithms, such as the well known Belief Propagation algorithm, have had much success in solving problems in statistical inference and coding and information theory. Survey Propagation attempts to apply iterative message passing algorithms to solve difficult combinatorial problems, in particular constraint satisfaction problems such as k-sat and coloring problems. Intuition from stat...
متن کاملNegative Tree Reweighted Belief Propagation
We introduce a new class of lower bounds on the log partition function of a Markov random field which makes use of a reversed Jensen’s inequality. In particular, our method approximates the intractable distribution using a linear combination of spanning trees with negative weights. This technique is a lower-bound counterpart to the tree-reweighted belief propagation algorithm, which uses a conv...
متن کاملLagrangian Relaxation for Inference in Natural Language Processing
There has been a long history in combinatorial optimization of methods that exploit structure in complex problems, using methods such as dual decomposition or Lagrangian relaxation. These methods leverage the observation that complex inference problems can often be decomposed into efficiently solvable sub-problems. Thus far, however, these methods are not widely used in NLP. In this talk I will...
متن کامل